{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Supervised Learning Algorithms: Linear Support Vector Machines"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*In this template, only **data input** and **input/target variables** need to be specified (see \"Data Input & Variables\" section for further instructions). None of the other sections needs to be adjusted. As a data input example, .csv file from IBM Box web repository is used.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Libraries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Run to import the required libraries.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib notebook\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Data Input and Variables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Define the data input as well as the input (X) and target (y) variables and run the code. Do not change the data & variable names **['df', 'X', 'y']** as they are used in further sections.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Data Input\n",
    "# df = \n",
    "\n",
    "### Defining Variables  \n",
    "# X = \n",
    "# y = \n",
    "\n",
    "### Data Input Example \n",
    "df = pd.read_csv('https://ibm.box.com/shared/static/q6iiqb1pd7wo8r3q28jvgsrprzezjqk3.csv')\n",
    "\n",
    "X = df[['horsepower']]\n",
    "y = df['price']\n",
    "\n",
    "X_2 = df[['horsepower', 'normalized-losses']]\n",
    "y_2 = df['price']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. The Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Run to build the model.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Linear SVC, C = 1.000\n",
      "Accuracy of Linear SVC classifier on training set: 0.27\n",
      "Accuracy of Linear SVC classifier on test set: 0.00\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state = 0)\n",
    "\n",
    "#C parameter\n",
    "this_C = 1.0\n",
    "\n",
    "#model\n",
    "clf = SVC(kernel = 'linear', C=this_C).fit(X_train, y_train)\n",
    "print('Linear SVC, C = {:.3f}'.format(this_C))\n",
    "print('Accuracy of Linear SVC classifier on training set: {:.2f}'\n",
    "     .format(clf.score(X_train, y_train)))\n",
    "print('Accuracy of Linear SVC classifier on test set: {:.2f}'\n",
    "     .format(clf.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1. Linear SVM regularization: C parameter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Linear SVM with C = 1e-05\n",
      "Accuracy of Linear SVM classifier on training set: 0.08\n",
      "Accuracy of Linear SVM classifier on test set: 0.00\n",
      "\n",
      "Linear SVM with C = 100\n",
      "Accuracy of Linear SVM classifier on training set: 0.27\n",
      "Accuracy of Linear SVM classifier on test set: 0.00\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for this_C in [0.00001, 100]:\n",
    "    clf = SVC(kernel = 'linear', C=this_C).fit(X_train, y_train)\n",
    "    print('Linear SVM with C = {}'.format(this_C))\n",
    "    print('Accuracy of Linear SVM classifier on training set: {:.2f}'\n",
    "         .format(clf.score(X_train, y_train)))\n",
    "    print('Accuracy of Linear SVM classifier on test set: {:.2f}\\n'\n",
    "         .format(clf.score(X_test, y_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2. LinearSVC with M classes generates M one vs rest classifiers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Coefficients:\n",
      " [[-2.14396025e-01  8.42666479e-02]\n",
      " [-1.17382373e-02 -2.65144462e-03]\n",
      " [-2.50881992e-02  2.65386917e-03]\n",
      " [-1.61677700e-02 -4.31542442e-04]\n",
      " [-1.29296866e-02 -2.01693724e-03]\n",
      " [-1.66081379e-02 -9.12590959e-04]\n",
      " [-1.30616076e-02 -7.62856148e-04]\n",
      " [-2.49290194e-02  4.42599109e-03]\n",
      " [-9.83381629e-03 -5.10230399e-03]\n",
      " [-1.68464031e-02 -8.35901198e-04]\n",
      " [-2.78919731e-02  5.84844881e-03]\n",
      " [-6.36812068e-03 -6.71373644e-03]\n",
      " [-4.65289397e-03 -7.14568659e-03]\n",
      " [-6.60199135e-03 -5.15677778e-03]\n",
      " [-1.33483761e-02 -1.89186669e-03]\n",
      " [-1.19880856e-02  1.23360392e-02]\n",
      " [-1.41249931e-02 -2.08341604e-03]\n",
      " [-3.17240951e-03  1.03903547e-02]\n",
      " [-7.43591448e-05  9.68438389e-03]\n",
      " [-1.51057191e-02 -4.54332614e-04]\n",
      " [-1.25707877e-02 -2.12566598e-03]\n",
      " [-3.60300645e-03 -8.41033278e-03]\n",
      " [-1.00827917e-02 -4.32172697e-03]\n",
      " [-5.60104665e-03 -7.46541361e-03]\n",
      " [-2.45725160e-02  2.27565777e-03]\n",
      " [-6.99547020e-03 -5.29453577e-03]\n",
      " [-7.50729974e-03 -5.09490567e-03]\n",
      " [-3.24504011e-03 -1.04632317e-02]\n",
      " [-1.28763717e-02 -9.87222966e-04]\n",
      " [-6.07320581e-03 -8.16852041e-03]\n",
      " [-1.14403763e-02 -3.56539990e-03]\n",
      " [-2.54676635e-03 -8.81117306e-03]\n",
      " [-1.41243331e-02 -1.37035804e-03]\n",
      " [-1.04894179e-02 -2.51157592e-03]\n",
      " [-2.34506717e-02  5.23119610e-03]\n",
      " [-6.58648642e-03 -4.80296672e-03]\n",
      " [-6.19437987e-03 -5.47020849e-03]\n",
      " [-1.29508907e-02 -6.66307160e-04]\n",
      " [-1.46862189e-02 -1.36133299e-03]\n",
      " [-5.42030942e-03  1.08281515e-02]\n",
      " [-8.26091254e-03 -5.53571258e-03]\n",
      " [-7.66136228e-03 -4.89077984e-03]\n",
      " [-5.49772048e-03 -7.01275571e-03]\n",
      " [-6.61433720e-03 -6.73789908e-03]\n",
      " [-4.42695314e-03 -7.30171625e-03]\n",
      " [-3.04956992e-02  7.10798933e-03]\n",
      " [-5.85170608e-02  1.86262800e-02]\n",
      " [-5.59840492e-03 -7.05964138e-03]\n",
      " [-6.99770750e-03 -4.44528750e-03]\n",
      " [-1.27172386e-02 -4.28149541e-04]\n",
      " [-1.29193902e-02 -2.34219412e-03]\n",
      " [-1.01739989e-02 -4.71094162e-03]\n",
      " [-3.04443035e-03 -6.63552890e-03]\n",
      " [-1.35599853e-03 -9.95826962e-03]\n",
      " [ 1.05448520e-03 -1.52497983e-02]\n",
      " [-6.24920490e-03 -7.41309348e-03]\n",
      " [-4.42914032e-03 -7.86928949e-03]\n",
      " [-7.66454752e-03 -4.25159517e-03]\n",
      " [-7.31605833e-03 -3.53749412e-03]\n",
      " [-7.00465766e-03 -5.06211133e-03]\n",
      " [-8.71377324e-03 -3.38372357e-03]\n",
      " [-1.11287485e-02 -2.99285388e-03]\n",
      " [-6.59286070e-03 -6.49724262e-03]\n",
      " [-5.36317851e-03 -6.93586317e-03]\n",
      " [-5.60898206e-03 -6.52954538e-03]\n",
      " [-5.90987143e-03 -6.12003125e-03]\n",
      " [-6.09953582e-03 -6.71876601e-03]\n",
      " [-1.57286444e-01  5.42901625e-02]\n",
      " [ 3.19573943e-03 -1.85483445e-02]\n",
      " [-7.71413066e-03 -6.01581538e-03]\n",
      " [-6.51019805e-03 -5.64191123e-03]\n",
      " [-8.83430764e-03 -4.56927425e-03]\n",
      " [-6.69844756e-04 -1.02585488e-02]\n",
      " [-2.13128021e-03 -9.67105366e-03]\n",
      " [-9.05475948e-03 -5.19140124e-03]\n",
      " [ 8.61079610e-04 -1.67023602e-02]\n",
      " [-8.66844150e-03 -3.75916976e-03]\n",
      " [-6.89763057e-03 -4.70872459e-03]\n",
      " [-9.97967547e-03 -4.21565130e-03]\n",
      " [-3.02611832e-04 -1.26405448e-02]\n",
      " [-2.83870866e-03 -1.07118996e-02]\n",
      " [-7.40407867e-03 -5.55672738e-03]\n",
      " [-1.19757403e-02 -2.25919117e-03]\n",
      " [-5.13761390e-03 -6.85032416e-03]\n",
      " [-1.02962746e-02 -3.43697459e-03]\n",
      " [-3.62255103e-03 -8.55366780e-03]\n",
      " [-1.51061530e-03 -1.23978298e-02]\n",
      " [ 7.86857423e-04 -1.30321535e-02]\n",
      " [-6.83083311e-04 -1.40166734e-02]\n",
      " [-2.79258959e-03 -1.04487412e-02]\n",
      " [-1.16856500e-02 -1.52966488e-03]\n",
      " [-2.55972446e-03 -1.09740127e-02]\n",
      " [ 4.39861013e-03 -1.86212443e-02]\n",
      " [-5.22265914e-03 -6.83751272e-03]\n",
      " [ 4.64324265e-04 -1.39573899e-02]\n",
      " [-1.02126926e-02 -4.12281740e-03]\n",
      " [-1.10669241e-02 -2.18740278e-03]\n",
      " [-2.78613015e-03  8.09413145e-03]\n",
      " [ 1.71666488e-03 -1.71445757e-02]\n",
      " [-1.33500081e-03 -1.08136856e-02]\n",
      " [-5.33720757e-03 -6.83340622e-03]\n",
      " [-1.15343213e-02 -2.61764653e-03]\n",
      " [ 8.80104659e-03 -3.24897962e-02]\n",
      " [-3.03489303e-03 -9.31547554e-03]\n",
      " [-6.61707597e-03 -6.71709528e-03]\n",
      " [-2.66395778e-02  7.56858869e-03]\n",
      " [-1.09366213e-03 -9.29695361e-03]\n",
      " [-4.85258990e-04 -1.39407479e-02]\n",
      " [ 4.12234642e-03 -1.82806261e-02]\n",
      " [-6.80462932e-03 -4.66299049e-03]\n",
      " [-9.88867572e-03 -2.68646108e-03]\n",
      " [-1.00024630e-02 -3.83673441e-03]\n",
      " [-1.22635736e-03 -1.08491021e-02]\n",
      " [-1.10121437e-02 -3.11299716e-03]\n",
      " [-1.80993941e-02  3.34098339e-04]\n",
      " [-3.85633237e-03 -8.83564642e-03]\n",
      " [-8.63796232e-03 -5.50291310e-03]\n",
      " [-4.34400676e-03 -6.36634295e-03]\n",
      " [-2.13682496e-03 -1.15543806e-02]\n",
      " [-8.06640570e-03 -4.06879875e-03]\n",
      " [ 3.51706540e-03 -2.07316261e-02]\n",
      " [ 3.06854737e-03 -1.77404008e-02]\n",
      " [ 3.32589323e-02 -6.88808017e-02]\n",
      " [ 1.27295521e-02 -3.30280315e-02]\n",
      " [-1.76104178e-03 -8.82591129e-03]\n",
      " [-5.37159351e-03 -6.20607657e-03]\n",
      " [ 1.53382641e-03 -1.64067923e-02]\n",
      " [-3.34038363e-03 -8.55443995e-03]\n",
      " [-2.61047796e-03 -1.11025386e-02]\n",
      " [-1.53337737e-03 -1.00464040e-02]\n",
      " [ 1.22716544e-02 -5.23036614e-03]\n",
      " [-1.93853798e-03 -1.21084622e-02]\n",
      " [-7.81190224e-04 -1.35544584e-02]\n",
      " [ 1.90665924e-05 -1.30879064e-02]\n",
      " [ 1.04533406e-02 -2.96454993e-02]\n",
      " [-1.81622660e-03 -1.06519005e-02]\n",
      " [-2.27613192e-03 -1.07474677e-02]\n",
      " [ 1.20922516e-02 -3.97719005e-02]\n",
      " [-2.56049890e-04 -1.43044696e-02]\n",
      " [ 8.60548143e-04 -1.58779161e-02]]\n",
      "Intercepts:\n",
      " [ 0.01254532 -0.01454626 -0.01742132 -0.00252403 -0.0219357  -0.02337793\n",
      " -0.01586994 -0.02287372  0.00462123 -0.02079588 -0.00366046  0.02279043\n",
      " -0.0233296   0.00046263 -0.02249522 -0.01843829 -0.01764555 -0.03075016\n",
      " -0.01540735 -0.02104688 -0.03006956  0.01809862 -0.04031767 -0.00578173\n",
      "  0.00217753 -0.02792996 -0.01019084 -0.02167765 -0.02163611 -0.01680886\n",
      " -0.01798087 -0.01957478 -0.021573   -0.00600343 -0.02241012 -0.0506601\n",
      " -0.009647   -0.02082834  0.01308944 -0.02199827 -0.03231453 -0.04597162\n",
      " -0.01635609 -0.01548489 -0.02525962 -0.02066032 -0.01925746 -0.00933718\n",
      " -0.05496637 -0.03550692 -0.04130853  0.00032973 -0.05029722 -0.03640648\n",
      "  0.00937706 -0.03975363 -0.02762816 -0.00993848 -0.09582902 -0.04863484\n",
      " -0.05534066 -0.0555024  -0.03867695 -0.05522551 -0.04797136 -0.05834683\n",
      " -0.03622433 -0.01635038  0.00950989 -0.05647114 -0.03288603 -0.03548065\n",
      " -0.01935263 -0.04079287 -0.03721051  0.00887693 -0.05078675 -0.04107982\n",
      " -0.05497602  0.00954188 -0.041121   -0.05613876 -0.05085163 -0.05580706\n",
      " -0.0478918  -0.05325284 -0.05366367 -0.0423538  -0.02429554 -0.06294562\n",
      " -0.04611254 -0.05036665 -0.01003085 -0.05061034 -0.04175053 -0.05127845\n",
      " -0.01984979 -0.05024378 -0.04189298 -0.0543486  -0.04276961 -0.04568472\n",
      " -0.01872168 -0.04196594 -0.07080196 -0.03628768 -0.09213109 -0.04748736\n",
      " -0.00951601 -0.07047381 -0.04788268 -0.04483294 -0.03499559 -0.04654704\n",
      " -0.04621714 -0.07035922 -0.05542719 -0.11588529 -0.05119463 -0.06921828\n",
      " -0.03411805 -0.03606885  0.01110691 -0.02331986 -0.0733811  -0.05888178\n",
      " -0.03363236 -0.03436679 -0.03638426 -0.05357524 -0.03252641 -0.03277087\n",
      " -0.04527914 -0.03301596 -0.03374925 -0.05100191 -0.06174497 -0.01300505\n",
      " -0.04467762 -0.04521454]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import LinearSVC\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X_2, y_2, random_state = 0)\n",
    "\n",
    "clf = LinearSVC(C=5, random_state = 67).fit(X_train, y_train)\n",
    "print('Coefficients:\\n', clf.coef_)\n",
    "print('Intercepts:\\n', clf.intercept_)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
